<!DOCTYPE html>
<html>
  
<head><meta name="generator" content="Hexo 3.9.0">
  <meta charset="utf-8">
  <meta name="author" content="Aa Y">
  
  
  <title>拯救被压爆的表情包 | Yaa&#39;s Bolg</title>

  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">

  
    <meta name="keywords" content="CV,">
  

  
  <meta name="description" content="各种论坛以及即时通讯工具通常会将表情包过分压缩，影响观感。本文参考EDSR(Enhanced Deep Super-Resolution)提出一种简单的表情包修复方法。">

  

  
    <script src="//cdn.jsdelivr.net/npm/leancloud-storage@3.11.1/dist/av-min.js" async></script>
  

  

  
    <script src="//unpkg.com/valine/dist/Valine.min.js" async></script>
  

  

  <script>
  // theme-ad's config script
  // it can be used in every script
  
  window.AD_CONFIG = {
    leancloud: {"appid":"fmmQ5HHXBeCLVf0DYXAfyBnM-gzGzoHsz","appkey":"WaRxtuWpVWcnAiqd2erlT4vy","comment":true,"count":true},
    welcome: {"enable":false,"interval":30},
    start_time: "2019-05-01",
    passwords: ["efe07af7441da2b69c4a41e42e73be4db47f66010a56900788a458354a7373ec", ],
    is_post: true,
    lock: false,
    author: "Aa Y",
    share: {"twitter":false,"facebook":false,"weibo":true,"qq":true,"wechat":true},
    mathjax: false,
    page_type: "",
    root: "/blog/"
  };
</script>

  <script src="/blog/vendor/sha256.min.js"></script>
<script src="/blog/js/auth.js"></script>
<script src="/blog/js/index.js"></script>
<script src="/blog/vendor/qrcode.min.js"></script>

  
    <link rel="icon" href="/blog/images/favicon.ico">
    <link rel="apple-touch-icon" href="/blog/images/touch-icon.png">
  

  <link href="//netdna.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css" rel="stylesheet">
  <link href="https://cdn.bootcss.com/KaTeX/0.7.1/katex.min.css" rel="stylesheet">

  <link rel="stylesheet" href="/blog/css/index.css">
<link rel="stylesheet" href="/blog/styles/components/highlight/highlight.css">

  
</head>
  <body>
    <header class="site-header">
  <div class="site-header-brand">
    
      <span class="site-header-brand-title">
        <a href="/blog/">Yaa</a>
      </span>
    
    
      <span class="site-header-brand-motto"> | Blog</span>
    
  </div>
  <div class="site-header-right">
    <nav class="site-header-navigation">
      
        <a href="/blog/" target="_self">首页</a>
      
        <a href="/blog/archives/" target="_self">归档</a>
      
        <a href="/blog/tags/" target="_self">标签</a>
      
        <a href="/blog/categories/" target="_self">分类</a>
      
        <a href="/blog/about/" target="_self">关于</a>
      
    </nav>
    <div class="site-header-btn">
      
        <a href="https://github.com/zyayoung/" target="_blank" id="site-github">
          <i class="fa fa-github-alt"></i>
        </a>
      
      <a href="javascript:void(0);" id="site-search">
        <i class="fa fa-search"></i>
      </a>
      <a href="javascript:void(0);" id="site-nav-btn">
        <i class="fa fa-ellipsis-v"></i>
      </a>
    </div>
  </div>
</header>
<nav class="table-content" id="site-nav">
  <div class="table-content-title">
    <span>导航</span>
  </div>
  <div class="table-content-main">
    <ol class="toc">
      
        <li class="toc-item">
          <a href="/blog/" target="_self">
            首页
          </a>
        </li>
      
        <li class="toc-item">
          <a href="/blog/archives/" target="_self">
            归档
          </a>
        </li>
      
        <li class="toc-item">
          <a href="/blog/tags/" target="_self">
            标签
          </a>
        </li>
      
        <li class="toc-item">
          <a href="/blog/categories/" target="_self">
            分类
          </a>
        </li>
      
        <li class="toc-item">
          <a href="/blog/about/" target="_self">
            关于
          </a>
        </li>
      
    </ol>
  </div>
</nav>
<div id="site-process"></div>
    <main>
      
  <div class="passage">
  <div class="passage-meta">
    <span>
      <i class="fa fa-calendar"></i>2019-10-04
    </span>
    
      <span>
        | <a href="/blog/categories/CV/"><i class="fa fa-bookmark"></i>CV</a>
      </span>
    
    
      <span>
        | <i class="fa fa-unlock-alt"></i>UNLOCK
      </span>
    
  </div>
  <h1 class="passage-title">
    拯救被压爆的表情包
  </h1>
  
  <article class="passage-article">
    <h1 id="简介"><a class="markdownIt-Anchor" href="#简介"></a> 简介</h1>
<p>JPG图像过度压缩后会出现明显的块状和波纹状噪音，本文提供一种修复噪音的方法。</p>
<p><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_resnet18srhead.jpg" alt></p>
<p>如图，左边的高质量表情包经过压缩后出现了噪音。经过Resnet18网络降噪，表情包的噪音被修复，观感与原图相近，甚至原图中被压缩的痕迹也得到了修复。</p>
<p>单图像超分辨率(SISR)问题已经有了很多研究。图像压缩修复是一个与之类似的问题：SISR重建降低分辨率所丢失的信息，而本文重建图像在过度压缩中丢失的信息。EDSR使用单网络实现图像超分辨率，是<em>NTIRE 2017</em>超分辨率挑战的第一名。本文使用与之类似的网络结构以实现图像压缩修复。</p>
<p><a href="https://github.com/zyayoung/FixJPG" target="_blank" rel="noopener">Code</a></p>
<table>
<thead>
<tr>
<th>Compressed</th>
<th>SRCNN修复</th>
<th>Resnet18修复</th>
</tr>
</thead>
<tbody>
<tr>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/xh.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_srcnnh.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_resnet18srh.jpg" alt></td>
</tr>
</tbody>
</table>
<h1 id="related-works"><a class="markdownIt-Anchor" href="#related-works"></a> Related Works</h1>
<h2 id="resnet"><a class="markdownIt-Anchor" href="#resnet"></a> Resnet</h2>
<p>Resnet引入残差框架使得深度网络更容易学习[2]。本文使用修改过的18层无pooling残差网络作为超分辨率基础网络。</p>
<h2 id="单图像超分辨率"><a class="markdownIt-Anchor" href="#单图像超分辨率"></a> 单图像超分辨率</h2>
<p>SRCNN[3]是早期的单图像超分辨率网络。网络输入大小和输出大小除padding外完全一致。训练时将训练图片双立方插值到1/2大小再双立方插值到原大小作为网络输入，原图作为网络输出。使用时需要先将图像双立方插值到2倍大小，再经过网络获得清晰的图像。我们仿照SRCNN的做法，训练时从训练图像中提取出64*64的黑白图像作为网络输入，以加速训练。测试时使用原图像大小。</p>
<p>在JPG压缩修复的问题中，我们不使用双立法插值准备训练数据，而使用高质量的训练图片过度压缩，得到训练样本。</p>
<p>EDSR是较新的单网络超分辨率方法。它移除了残差网络中多余的层，达到了2017年的state-of-the-art[1]。 该文章提出了Residual Module中的Batch Normalization层&quot;消除了范围的灵活性&quot;[1]，对超分辨率任务无利。本文中重建图像的任务与超分辨率类似，使用无Batch Normalization的残差网络。</p>
<h1 id="proposed-methods"><a class="markdownIt-Anchor" href="#proposed-methods"></a> Proposed Methods</h1>
<h2 id="data-preparation"><a class="markdownIt-Anchor" href="#data-preparation"></a> Data preparation</h2>
<p>我们在百度图片中以&quot;表情包&quot;为关键词找到700张图片，并且人工筛选出209张高质量图片作为训练数据集。在训练数据集中，我们对每张图片使用python-pillow进行压缩，并且将图像长边限制在320px，得到网络的输入和输出数据。对于网路输入数据，选择压缩质量10。对于网路输出数据，选择压缩质量100。</p>
<h2 id="model"><a class="markdownIt-Anchor" href="#model"></a> Model</h2>
<p>本文提出两种可用于JPG压缩修复的网络结构：</p>
<h3 id="srcnn"><a class="markdownIt-Anchor" href="#srcnn"></a> SRCNN</h3>
<p><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/srcnn.jpg" alt></p>
<p>使用SRCNN中提出的网络结构：</p>
<p>Conv(64 kernals, 9x9) -&gt; relu -&gt; Conv(32 kernals, 1x1) -&gt; relu -&gt; Conv(1 kernal, 5x5)</p>
<h3 id="resnet18"><a class="markdownIt-Anchor" href="#resnet18"></a> Resnet18</h3>
<p><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/resnet18sr.jpg" alt></p>
<p>使用类似EDSR的设计，去除resnet18的stride，保留8个同样的ResBlock。保留了最后的add层，以迫使网络习得由JPG压缩所产生的噪音。网络共18个卷积层。如图所示，ResBlock与EDSR中略有不同，保留了最后的relu层。除可根据上下文推断的情况外，所有卷积均为3x3, 64filters。</p>
<h2 id="training"><a class="markdownIt-Anchor" href="#training"></a> Training</h2>
<p>由于训练表情包多数以黑白为主，训练时先将图片转为灰度图，再从中截出64*64的部分图片送入网络，使用Adam优化器keras默认参数，batchsize=256，训练24epochs。</p>
<h2 id="inference"><a class="markdownIt-Anchor" href="#inference"></a> Inference</h2>
<p>对于黑白图片直接通过网络得到修复后的图像。对于彩色图片，每个通道分别通过网络后，合并得到修复后的图像。可对输出图像再次进入网络，可能会得到更清晰的图像。</p>
<h1 id="experiments"><a class="markdownIt-Anchor" href="#experiments"></a> Experiments</h1>
<p>使用两种网络在训练集上测试图像修复效果</p>
<table>
<thead>
<tr>
<th></th>
<th>Raw</th>
<th>SRCNN</th>
<th>Resnet18</th>
</tr>
</thead>
<tbody>
<tr>
<td>PSNR</td>
<td>27.52</td>
<td>30.00</td>
<td>31.31</td>
</tr>
<tr>
<td>Sample1</td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/x.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_srcnn.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_resnet18sr.jpg" alt></td>
</tr>
<tr>
<td>Sample2</td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/x1.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_srcnn1.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_resnet18sr1.jpg" alt></td>
</tr>
<tr>
<td>Sample3</td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/x2.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_srcnn2.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_resnet18sr2.jpg" alt></td>
</tr>
<tr>
<td>Sample3</td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/x3.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_srcnn3.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_resnet18sr3.jpg" alt></td>
</tr>
<tr>
<td>Sample4</td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/xh.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_srcnnh.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/y_resnet18srh.jpg" alt></td>
</tr>
</tbody>
</table>
<p>Table 1: Average PSNR of Reconstruction on the 209-Image Dataset</p>
<table>
<thead>
<tr>
<th>SRCNN</th>
<th>Resnet18</th>
</tr>
</thead>
<tbody>
<tr>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_srcnn.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_resnet18sr.jpg" alt></td>
</tr>
<tr>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_srcnn1.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_resnet18sr1.jpg" alt></td>
</tr>
<tr>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_srcnn2.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_resnet18sr2.jpg" alt></td>
</tr>
<tr>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_srcnn3.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_resnet18sr3.jpg" alt></td>
</tr>
<tr>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_srcnnh.jpg" alt></td>
<td><img src="https://raw.githubusercontent.com/zyayoung/FixJPG/master/demo/demo_resnet18srh.jpg" alt></td>
</tr>
</tbody>
</table>
<p>Table 2: Comparison between Two Models. From top to down: Raw image, fixed image, Residual (noise).</p>
<h1 id="reference"><a class="markdownIt-Anchor" href="#reference"></a> Reference</h1>
<ol>
<li>B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, “Enhanced Deep Residual Networks for Single Image Super-Resolution,” <em>2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)</em>, 2017.</li>
<li>K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” <em>2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)</em>, 2016.</li>
<li>C. Dong, C. C. Loy, K. He, and X. Tang.  “Learning a deep convolutional network for image super-resolution,” <em>Proceedings of European Conference on Computer Vision (ECCV)</em>, 2014.</li>
</ol>
  </article>
  <aside class="table-content" id="site-toc">
  <div class="table-content-title">
    <i class="fa fa-arrow-right fa-lg" id="site-toc-hide-btn"></i>
    <span>目录</span>
  </div>
  <div class="table-content-main">
    <ol class="toc"><li class="toc-item toc-level-1"><a class="toc-link" href="#简介"><span class="toc-text"> 简介</span></a></li><li class="toc-item toc-level-1"><a class="toc-link" href="#related-works"><span class="toc-text"> Related Works</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#resnet"><span class="toc-text"> Resnet</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#单图像超分辨率"><span class="toc-text"> 单图像超分辨率</span></a></li></ol></li><li class="toc-item toc-level-1"><a class="toc-link" href="#proposed-methods"><span class="toc-text"> Proposed Methods</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#data-preparation"><span class="toc-text"> Data preparation</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#model"><span class="toc-text"> Model</span></a><ol class="toc-child"><li class="toc-item toc-level-3"><a class="toc-link" href="#srcnn"><span class="toc-text"> SRCNN</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#resnet18"><span class="toc-text"> Resnet18</span></a></li></ol></li><li class="toc-item toc-level-2"><a class="toc-link" href="#training"><span class="toc-text"> Training</span></a></li><li class="toc-item toc-level-2"><a class="toc-link" href="#inference"><span class="toc-text"> Inference</span></a></li></ol></li><li class="toc-item toc-level-1"><a class="toc-link" href="#experiments"><span class="toc-text"> Experiments</span></a></li><li class="toc-item toc-level-1"><a class="toc-link" href="#reference"><span class="toc-text"> Reference</span></a></li></ol>
  </div>
</aside>
  
    <aside class="passage-copyright">
      <div>本文作者: Yaa</div>
      
        <div>
          原文链接: 
          <a href="" target="_blank">http://zyayoung.gitee.io/blog/passages/拯救被压爆的表情包/</a>
        </div>
      
      <div>
        转载请注明出处!
      </div>
    </aside>
  
  
</div>

    </main>
    
      
<div class="site-comment-contanier" data-plateform="leancloud">
  
    <p id="site-comment-info">
      <i class="fa fa-spinner fa-spin"></i> 评论加载中
    </p>
    <div id="site-comment"></div>
  
</div>
    
    <div class="site-footer-wrapper">
  <footer class="site-footer">
    
      
        <div class="site-footer-col">
          <h5 class="site-footer-title">博客推荐</h5>
          
            <span class="site-footer-item">
              <a href="https://scarlet-climax.github.io/" target="_blank">Scarlet Devil Mansion</a>
            </span>
          
        </div>
      
    
    <div class="site-footer-info">
      <i class="fa fa-clock-o"></i> 本站已稳定运行<span id="site-time"></span>
    </div>
    
      <div class="site-footer-info">
        <i class="fa fa-paw"></i> 您是本站第 <span id="site-count"></span> 位访客
      </div>
    
    
      <div class="site-footer-info">
        <i class="fa fa-at"></i> Email: 1328410180@qq.com
      </div>
    
    <div class="site-footer-info">
      <i class="fa fa-copyright"></i> 
      2019 <a href="https://github.com/dongyuanxin/theme-ad/" target="_blank">Theme-AD</a>.
      Created by <a href="https://godbmw.com/" target="_blank">GodBMW</a>.
      All rights reserved.
    </div>
  </footer>
</div>
    <div id="site-layer" style="display:none;">
  <div class="site-layer-content">
    <div class="site-layer-header">
      <span class="site-layer-header-title" id="site-layer-title"></span>
      <i class="fa fa-close" id="site-layer-close"></i>
    </div>
    <div class="site-layer-body" id="site-layer-container">
      <div class="site-layer-input" id="site-layer-search" style="display: none;">
        <input type="text">
        <i class="fa fa-search"></i>
      </div>
      
      <div id="site-layer-welcome" style="display:none;"></div>
    </div>
  </div>
</div>
    

<div class="bottom-bar">
  <div class="bottom-bar-left">
    <a href="/blog/passages/拯救被压爆的表情包Demo/" data-enable="true">
      <i class="fa fa-arrow-left"></i>
    </a>
    <a href="/blog/passages/Detection/" data-enable="true">
      <i class="fa fa-arrow-right"></i>
    </a>
  </div>
  <div class="bottom-bar-right">
    <a href="javascript:void(0);" data-enable="true" id="site-toc-show-btn">
      <i class="fa fa-bars"></i>
    </a>
    
      <a href="#site-comment" data-enable="true">
        <i class="fa fa-commenting"></i>
      </a>
    
    <a href="javascript:void(0);" id="site-toggle-share-btn">
      <i class="fa fa-share-alt"></i>
    </a>
    
    <a href="javascript:void(0);" id="back-top-btn">
      <i class="fa fa-chevron-up"></i>
    </a>
  </div>
</div>
    <div id="share-btn">
  
  
  
    <a id="share-btn-weibo" href="javascript:void(0);" target="_blank">
      <i class="fa fa-weibo"></i>
    </a>
  
  
    <a id="share-btn-qq" href="javascript:void(0);" target="_blank">
      <i class="fa fa-qq"></i>
    </a>
  
  
    <a id="share-btn-wechat" href="javascript:void(0);" target="_blank">
      <i class="fa fa-wechat"></i>
    </a>
  
</div>
    





    
  </body>
</html>